The present invention is generally directed to systems and methods for evaluating video quality, and, in particular, to an improved system and method for providing a scalable dynamic objective metric for automatically evaluating video quality of a video image.
Video experts continually seek new algorithms and methods for improving the quality of video images. The primary goal is to obtain the most perceptually appealing video image possible. The ultimate criterion is the question xe2x80x9cHow well does the viewer like the resulting picture?xe2x80x9d One way to answer the question is to have a panel of viewers watch certain video sequences and then record the opinions of the viewers concerning the resulting image quality. The results, however, will vary from panel to panel according to the variability between the viewing panels. This problem is commonly encountered when relying on subjective human opinion. The severity of the problem is increased when the viewing panel is composed of non-experts.
Results solely based upon on human perception and subjective opinion are usually subjected to subsequent statistical analysis to remove ambiguities that result from the non-deterministic nature of subjective results. Linear and non-linear heuristic statistical models have been proposed to normalize these types of subjective results and obtain certain figures of merit that represent the goodness (or the degradation) of video quality. The process of measuring video quality in this manner is referred to as xe2x80x9csubjective video quality assessment.xe2x80x9d
Subjective video quality assessment methods give valid indications of visible video artifacts. Subjective video quality assessment methods, however, are probabilistic in nature, complex, time consuming, and sometimes difficult to apply. In addition, there is a problem in selecting appropriate viewers for the viewing panel. A non-trained viewer will be a poor judge of the suitability of new video processing methods. A non-trained viewer, however, will likely accurately represent the general consumers in the marketplace. On the other hand, a trained expert viewer will be overly biased toward detecting minor defects that will never be noticed by the general consumer.
To avoid the disadvantages that attend subjective methods for evaluating video quality, it is desirable to use automated objective methods to evaluate video quality. Automated objective methods seek to obtain objective figure of merits to quantify the goodness (or the degradation) of video quality. The process for obtaining one or more objective measures of the video quality must be automated in order to be able to quickly analyze differing types of video algorithms as the video algorithms sequentially appear in a video stream.
Objective measures of video quality are fully deterministic. That is, the results will always be the same when the test is repeated (assuming the same settings are preserved).
Because the ultimate goal is to present the viewer with the most appealing picture, a final judge of the value of the objective measures of video quality is the degree of correlation that the objective measures have with the subjective results. Statistical analysis is usually used to correlate the results objectively obtained (automatically generated) with the results subjectively obtained (from human opinion).
There is a need in the art for improved systems and methods for automatically measuring video quality. The process of automatically measuring video quality is referred to as xe2x80x9cobjective video quality assessment.xe2x80x9d
Several different types of algorithms have been proposed that are capable of providing objective video quality assessment. The algorithms are generally referred to as xe2x80x9cobjective video quality models.xe2x80x9d A report from the Video Quality Experts Group (VQEG) sets forth and describes the results of an evaluation performed on ten (10) objective video quality models. The report is dated December 1999 and is entitled xe2x80x9cFinal Report from the Video Quality Experts Group on the validation of Objective Models of Video Quality Assessment.xe2x80x9d
Each different objective video quality model provides its own distinctive measurement of video quality referred to as an xe2x80x9cobjective metric.xe2x80x9d A xe2x80x9cdouble endedxe2x80x9d objective metric is one that evaluates video quality using a first original video image and a second processed video image. A xe2x80x9cdouble endedxe2x80x9d objective metric compares the first original video image to the second processed video image to evaluate video quality by determining changes in the original video image. A xe2x80x9csingle endedxe2x80x9d objective metric is one that evaluates video quality without referring to the original video image. A xe2x80x9csingle endedxe2x80x9d objective metricxe2x80x9d applies an algorithm to a video image to evaluate its quality.
No single objective metric has been found to be superior to all the other objective metrics under all conditions and for all video artifacts. Each objective metric has its own advantages and disadvantages. Objective metrics differ widely in performance (i.e., how well their results correlate with subjective quality assessment results), and in stability (i.e., how well they handle different types of video artifacts), and in complexity (i.e., how much computation power is needed to perform the algorithm calculations).
A wide range of applications exist to which objective metrics may be applied. For example, fast real-time objective metrics are needed to judge the quality of a broadcast video signal. On the other hand, more complex and reliable objective metrics are better for judging the quality of non-real time video simulations.
Using only one objective metric (and one objective video quality model) limits the evaluation of the quality of a video signal to the level of evaluation that is obtainable from the objective metric that is used. There is a need in the art for an improved system and method that uses more than one objective metric for video quality evaluation.
The present invention generally comprises an improved system and method for providing a scalable dynamic objective metric for automatically evaluating video quality of a video image.
In an advantageous embodiment of the present invention, the improved system of the invention comprises an objective metric controller that is capable of receiving a plurality of objective metric figures of merit from a plurality of objective metric model units. The objective metric controller is capable of determining a scalable dynamic objective metric from the plurality of objective figures of merit.
In an advantageous embodiment of the present invention, the improved method of the invention comprises the steps of 1) receiving in an objective metric controller a plurality of objective metric figures of merit from a plurality of objective metric model units, and 2) determining a scalable dynamic objective metric from the plurality of said objective metric figures of merit.
It is a primary object of the present invention to provide an improved system and method for providing a scalable dynamic objective metric for automatically evaluating video quality of a video image.
It is another object of the present invention to provide a scalable dynamic objective metric by obtaining a weighted average of a plurality of objective metric figures of merit.
It is an additional object of the present invention to provide a scalable dynamic objective metric by obtaining a weighted average of a plurality of objective metric figures of merit using a correlation factor that represents how well an objective metric figure of merit evaluates video image characteristics.
It is another object of the present invention to continually determine new values of the scalable dynamic objective metric from new values of the plurality of objective metric figures of merit as new video images are continually received.
The foregoing has outlined rather broadly the features and technical advantages of the present invention so that those skilled in the art may better understand the Detailed Description of the Invention that follows. Additional features and advantages of the invention will be described hereinafter that form the subject of the claims of the invention. Those skilled in the art should appreciate that they may readily use the conception and the specific embodiment disclosed as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. Those skilled in the art should also realize that such equivalent constructions do not depart from the spirit and scope of the invention in its broadest form.
Before undertaking the Detailed Description of the Invention, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document: the terms xe2x80x9cincludexe2x80x9d and xe2x80x9ccomprisexe2x80x9d and derivatives thereof, mean inclusion without limitation; the term xe2x80x9cor,xe2x80x9d is inclusive, meaning and/or; the phrases xe2x80x9cassociated withxe2x80x9d and xe2x80x9cassociated therewith,xe2x80x9d as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term xe2x80x9ccontroller,xe2x80x9d xe2x80x9cprocessor,xe2x80x9d or xe2x80x9capparatusxe2x80x9d means any device, system or part thereof that controls at least one operation, such a device may be implemented in hardware, firmware or software, or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases.